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CN , Abstract 

■ This paper considers the problem of simultaneously communicating two messages, a 

I . high-security message and a low-security message, to a legitimate receiver, referred to 

I as the security embedding problem. An information-theoretic formulation of the prob- 

lem is presented. A coding scheme that combines rate splitting, superposition coding, 
nested binning and channel prefixing is considered and is shown to achieve the secrecy 
capacity region of the channel in several scenarios. Specifying these results to both scalar 
c/5 ' and independent parallel Gaussian channels (under an average individual per-subchannel 

power constraint), it is shown that the high-security message can be embedded into the 
low-security message at full rate (as if the low-security message does not exist) without 
^ incurring any loss on the overall rate of communication (as if both messages are low- 

iC) • security messages). Extensions to the wiretap channel II setting of Ozarow and Wyner 

, are also considered, where it is shown that "perfect" security embedding can be achieved 

by an encoder that uses a two-level coset code. 
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1 Introduction 



Physical layer security has been a very active area of research in information theory. See [T] 
^ I and [2] for overviews of recent progress in this field. A basic model of physical layer security is a 
wiretap/broadcast channel [3lll] with two receivers, a legitimate receiver and an eavesdropper. 
Both the legitimate receiver and the eavesdropper channels are assumed to be known at the 
transmitter. By exploring the (statistical) difference between the legitimate receiver channel 
and the eavesdropper channel, one may design coding schemes that can deliver a message 
reliably to the legitimate receiver while keeping it asymptotically perfectly secret from the 
eavesdropper. 

While assuming the transmitter's knowledge of the legitimate receiver channel might be 
reasonable (particularly when a feedback link is available), assuming that the transmitter 
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knows the eavesdropper channel is unrealistic in most scenarios. This is mainly because the 
eavesdropper is an adversary, who usually has no incentive to help the transmitter to acquire 
its channel state information. Hence, it is critical that physical layer security techniques are 
designed to withstand the uncertainty of the eavesdropper channel. 

In this paper, we consider a communication scenario where there are multiple possible 
realizations for the eavesdropper channel. Which realization will actually occur is unknown 
to the transmitter. Our goal is to design coding schemes such that the number of secure 
bits delivered to the legitimate receiver depends on the actual realization of the eavesdropper 
channel. More specifically, when the eavesdropper channel realization is weak, all bits delivered 
to the legitimate receiver need to be secure. In addition, when the eavesdropper channel 
realization is strong, a prescribed part of the bits needs to remain secure. We call such 
codes security embedding codes, referring to the fact that high-security bits are now embedded 
into the low-security ones. We envision that such codes are naturally useful for the secrecy 
communication scenarios where the information bits are not created equal: some of them have 
more security priorities than the others and hence require stronger security protections during 
communication. For example, in real wireless communication systems, control plane signals 
have higher secrecy requirement than data plane transmissions, and signals that carry users' 
identities and cryptographic keys require stronger security protections than the other signals. 

A key question that we consider is at what expense one may allow part of the bits to 
enjoy stronger security protections. Note that a "naive" security embedding scheme is to 
design two separate secrecy codes to provide two different levels of security protections, and 
apply them to two separate parts of the information bits. In this scheme, the high-security 
bits are protected using a stronger secrecy code and hence are communicated at a lower rate. 
The overall communication rate is a convex combination of the low-security bit rate and the 
high-security bit rate and hence is lower than the low-security bit rate. Moreover, this rate 
loss becomes larger as the portion of the high-security bits becomes larger and the additional 
security requirement (for the high-security bits) becomes higher. 

The main result of this paper is to show that it is possible to have a significant portion 
of the information bits enjoying additional security protections without sacrificing the overall 
communication rate. This further justifies the name "security embedding," as having part of 
the information bits enjoying additional security protections is now only an added bonus. More 
specifically, in this paper, we call a secrecy communication scenario embeddable if a nonzero 
fraction of the information bits can enjoy additional security protections without sacrificing 
the overall communication rate, and we call it perfectly embeddable if the high-security bits can 
be communicated at full rate (as if the low-security bits do not exist) without sacrificing the 
overall communication rate. Key to achieve optimal security embedding is to jointly encode the 
low-security and high-security bits (as opposed to separate encoding as in the naive scheme). 
In particular, the low-security bits can be used as (part of) the transmitter randomness to 
protect the high-security bits (when the eavesdropper channel realization is strong); this is a 
key feature of our proposed security embedding codes. 

The rest of the paper is organized as follows. In Sec. [21 we briefly review some basic results 
on the secrecy capacity and optimal encoding scheme for several classical wiretap channel set- 
tings. These results provide performance and structural benchmarks for the proposed security 
embedding codes. In Sec. |3l an information-theoretic formulation of the security embedding 
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Figure 1: Wiretap channel. 



problem is presented, which we term as two-level security wiretap channel. A coding scheme 
that combines rate splitting, superposition coding, nested binning and channel prefixing is 
proposed and is shown to achieve the secrecy capacity region of the channel in several scenar- 
ios. Based on the results of Sec. O in Sec. H] we study the engineering communication models 
with real channel input and additive white Gaussian noise, and show that both scalar and 
independent parallel Gaussian (under an individual per-subchannel average power constraint) 
two-level security wiretap channels are perfectly emheddahle. In Sec. El we extend the results 
of Sec. [3] to the wiretap channel II setting of Ozarow and Wyner [5], and show that two-level 
security wiretap channels II are also pefectly emheddahle. Finally, in Sec. |6l we conclude the 
paper with some remarks. 



2 Wiretap Channel: A Review 

Consider a discrete memoryless wiretap channel with transition probability p{y, z\x), where X 
is the channel input, and Y and Z are the channel outputs at the legitimate receiver and the 
eavesdropper, respectively (see Fig. [1]). The transmitter has a message M, uniformly drawn 
from {!,..., 2"^} where n is the block length and R is the rate of communication. The message 
M is intended for the legitimate receiver, but needs to be kept asymptotically perfectly secret 
from the eavesdropper. Mathematically, this secrecy constraint can be written as 

-J(M;Z'^)^0 (1) 

n 

in the limit as n — )■ cxo, where = (2'[1], . . . , Z[n]) is the collection of the channel outputs at 
the eavesdropper during communication. A communication rate R is said to be achievahle if 
there exists a sequence of codes of rate R such that the message M can be reliably delivered 
to the legitimate receiver while satisfying the asymptotic perfect secrecy constraint ([T]). The 
largest achievable rate is termed as the secrecy capacity of the channel. 

A discrete memoryless wiretap channel p{y, z\x) is said to be degraded if X ^ Y Z 
forms a Markov chain in that order. The secrecy capacity Cg of a degraded wiretap channel 
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was characterized by Wyner [3] and can be written as 

Cs = max [I{X; Y) - I{X; Z)] (2) 

p{x) 

where the maximization is over all possible input distributions p{x). The scheme proposed in 
[3] to achieve the secrecy capacity ([2]) is random binning, which can be described as follows. 

Consider a codebook of 2'^^^^'^'' codewords, each of length n. The codewords are partitioned 
into 2"^ bins, each containing 2^"^ codewords. Given a message m (which is uniformly drawn 
from {1, . . . , 2"'^}), the encoder randomly and uniformly chooses a codeword in the mth bin 
and sends it through the channel. The legitimate receiver needs to decode the entire codebook 
(and hence recover the transmitted message m), so the overall rate R + T cannot be too high. 
On the other hand, the rate T of the sub-codebooks in each bin represents the amount of 
external randomness injected by the transmitter (transmitter randomness) into the channel 
and hence needs to be sufficiently large to confuse the eavesdropper. With an appropriate 
choice of the codebooks and the partitions of bins, it was shown in [3] that any communication 
rate R less than the secrecy capacity ([2]) is achievable by the aforementioned random binning 
scheme. 

For a general discrete memoryless wiretap channel p{y,z\x) where the channel outputs Y 
and Z are not necessarily ordered, the random binning scheme of [3] is not necessarily optimal. 
In this case, the secrecy capacity Cg of the channel was characterized by Csiszar and Korner 
[1] and can be written as 

Cs = max [I{V; Y) - I{V; Z)] (3) 

p{v,x) 

where V is an auxiliary random variable satisfying the Markov chain V ^ X ^ [Y, Z) . The 
scheme proposed in [1] is to first prefix the channel input X hj V and view V as the input of the 
induced wiretap channel p{y,z\v) = ^^.p^/, 2|a;)p(x|f ). Applying the random binning scheme 
of [3] to the induced wiretap channel p{y, z\v) proves the achievability of rate I{V] Y) — I{V; Z) 
for any given joint auxiliary-input distribution p{v,x). 

In communication engineering, communication channels are usually modeled as discrete- 
time channels with real input and additive white Gaussian noise. Consider a (scalar) Gaussian 
wiretap channel where the channel outputs at the legitimate receiver and the eavesdropper are 
given by 

Y = v^X + iVi 

Z = VbX + N2. ^ ^ 

Here, X is the channel input which is subject to the average power constraint 

1 " 

-J2{x[^r<p (5) 

a and b are the channel gains for the legitimate receiver and the eavesdropper channel respec- 
tively, and A^i and N2 are additive white Gaussian noise with zero means and unit variances. 
The secrecy capacity of the channel was characterized in [6] and can be written as 



Cs{P,a,b) 



^log(l + aP)-^log(l + 6P) 



(6) 
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where [x]~^ := max(0,x). Note from that Cs{P.,a,b) > if and only if a > 6. That is, for 
the Gaussian wiretap channel (jl]), asymptotic perfect secrecy communication is possible if and 
only if the legitimate receiver has a larger channel gain than the eavesdropper. In this case, 
we can equivalently write the channel output Z at the eavesdropper as a degraded version of 
the channel output Y at the legitimate receiver, and the random binning scheme of [3] with 
Gaussian codebooks and full transmit power achieves the secrecy capacity of the channel. 

A closely related engineering scenario consists of a bank of L independent parallel scalar 
Gaussian wiretap channels [7]. In this scenario, the channel outputs at the legitimate receiver 
and the eavesdropper are given hj Y = (Yi, . . . , Yl) and Z = (Zi, . . . , Zl) where 



Yi = ^iXi + N,^i 
Zl = VbiXi + N2, 



/ = !,.. .,L. (7) 



Here, Xi is the channel input for the Ith subchannel, ai and bi are the channel gains for the 
legitimate receiver and the eavesdropper channel respectively in the /th subchannel, and A^^i ^ 
and N2^i are additive white Gaussian noise with zero means and unit variances. Furthermore, 
{Ni^i, N2^i) are independent for Z = 1, . . . , L so all L subchannels are independent of each other. 

Two different types of power constraints have been considered: the average individual 
per-subchannel power constraint 

n 

-5^(X,[2])2<F,, l = l,...,L (8) 

i=\ 

and the average total power constraint 



E 

1=1 



n 

i=l 



< P- (9) 



Under the average individual per-subchannel power constraint ([8]), the secrecy capacity of the 
independent parallel Gaussian wiretap channel ([7]) is given by |7] 

L 

Cs{{Pu ai, bi}ti) = Cs{Pi, ai, bi) (10) 
1=1 

where Cs{P,a,b) is defined as in Clearly, any communication rate less than the secrecy 
capacity fITU]) can be achieved by using L separate scalar Gaussian wiretap codes, each for one 
of the L subchannels. The secrecy capacity, Cs{P, {ai,bi}fL-^), under the average total power 
constraint (|9]) is given by 

L 

Cs{P,{ai,bi}t,)= max J]] C,(Pi, a^, 6;) (11) 
(Pi,...,Pi) 

where the maximization is over all possible power allocations (Pi, . . . , P^) such that Ylf=i Pi — 
P. A waterfilling-like solution for the optimal power allocation was derived in [TJ Th. 1], which 
provides an efficient way to numerically calculate the secrecy capacity Cs{P, {cii,bi}fLi). 
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3 Two-Level Security Wiretap Channel 



3.1 Channel Model 

Consider a discrete memoryless broadcast channel with three receivers and transition proba- 
bihty p{y, Zi, Z2\x). The receiver that receives the channel output y is a legitimate receiver. 
The receivers that receive the channel outputs Zi and Z2 are two possible realizations of an 
eavesdropper. Assume that the channel output Z2 is degraded with respect to the channel 
output Zi, i.e., 

X ^ Zi^ Z2 (12) 

forms a Markov chain in that order. Therefore, the receiver that receives the channel output 
Zi represents a stronger realization of the eavesdropper channel than the receiver that receives 
the channel output Z2. 

The transmitter has two independent messages: a high-security message Mi uniformly 
drawn from { 1 , . . . , 2'^^^ } and a low-security message M2 uniformly drawn from { 1 , . . . , 2"'^'^ } . 
Here, n is the block length, and Ri and R2 are the corresponding rates of communication. 
Both messages Mi and M2 are intended for the legitimate receiver, and need to be kept 
asymptotically perfectly secure when the eavesdropper realization is weak, i.e., 

-/(Mi,M2;Z^0 ^0 (13) 
n 

in the limit as n — ?■ 00. In addition, when the eavesdropper realization is strong, the high- 
security message Mi needs to remain asymptotically perfectly secure, i.e., 

-/(Mi; Zf) (14) 
n 

in the limit as n — t- 00. A rate pair (i?i, R2) is said to be achievable if there is a sequence of 
codes of rate pair (i?i, R2) such that both messages Mi and M2 can be reliably delivered to the 
legitimate receiver while satisfying the asymptotic perfect secrecy constraints f ll3p and fll4p . 
The collection of all possible achievable rate pairs is termed as the secrecy capacity region of 
the channel. Fig. [2] illustrates this communication scenario, which we term as two-level security 
wiretap channel. 

The above setting of two-level security wiretap channel is closely related to the traditional 
wiretap channel setting of [31 H]. More specifically, without the additional secrecy constraint 
f[T^ on the high-security message Mi, we can simply view the messages Mi and M2 as a single 
(low-security) message M with rate Ri + R2- And the problem reduces to communicating 
the message M over the traditional wiretap channel with transition probability p{y,Z2\x) = 
J2zi Piy^ -^1' ^2\x). By the secrecy capacity expression ([3]), the maximum achievable Ri + R2 is 
given by 

max[I{V;Y)- I{V;Z2)] (15) 

p{v,x) 

where V is an auxiliary random variable satisfying the Markov chain V ^ X ^ {Y, Z2). Simi- 
larly, without needing to communicate the low-security message M2 (i.e., R2 = 0), the secrecy 
constraint (HM reduces to (l/n)/(Mi; Z^) — ?■ which is implied by the secrecy constraint (fT3l) 
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Figure 2: Two-level security wiretap channel. 



since /(Mi; Z2 ) < 1{M\] Z"^) due to the Markov chain fll2l) . In this case, the problem reduces 
to communicating the high-security message Mi over the traditional wiretap channel with 
transition probability p{y, zi\x) = J2z2 P(y^ zi,Z2\x). Again, by the secrecy capacity expression 
([3]), the maximum achievable -Ri is given by 

mixx[I{W-X) - I{W;Zi)] (16) 

where W is an auxiliary random variable satisfying the Markov chain W ^ X ^ {Y, Zi). 

Based on the above connections, we may conclude that a two-level security wiretap channel 
p{y, Zi, Z2\x) is embeddable if there exists a sequence of coding schemes with a rate pair (_Ri, R2) 
such that -Ri + R2 is equal to f[T^ and -Ri > 0, and it is perfectly embeddable if there exists a 
sequence of coding schemes with a rate pair (_Ri,_R2) such that Ri + R2 is equal to (IT^ and 
Ri is equal to (fT6l) . 

An important special case of the two-level security wiretap channel problem considered 
here is when the channel output Z2 is a constant signal. In this case, the secrecy constraint 
(ITS]) becomes obsolete, and the low-security message M2 becomes a regular message without 
any secrecy constraint. The problem of simultaneously communicating a regular message and 
a confidential message over a discrete memoryless wiretap channel was first considered in [5], 
where a single-letter characterization of the capacity region was established. For the general 
two-level security wiretap channel problem that we consider here, both high-security message 
Ml and low-security message M2 are subject to asymptotic perfect secrecy constraints, which 
makes the problem potentially much more involved. 

3.2 Main Results 

The following theorem provides two sufficient conditions for establishing the achievability of 
a rate pair for a given discrete memoryless two-level security wiretap channel. 
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Theorem 1. Consider a discrete memoryless two-level security wiretap channel with transition 
probability p{y, Zi, Z2\x) that satisfies the Markov chain (fT2|) . A nonnegative pair (i?i,i?2) is 
an achievable rate pair of the channel if it satisfies 

Ri < J(X;F)-/(X;Zi) 
R1 + R2 < IiX;Y)-I{X;Z2) ^ ' 

for some input distribution p[x) . More generally, a nonnegative pair (i?i,i?2) is an achievable 
rate pair of the channel if it satisfies 

Ri < IiV;Y\U)-IiV;Z,\U) 
R1 + R2 < I{V;Y)-I{V;Z2) ^'"^ 

for some joint distribution p{u, v, x), where U and V are auxiliary random variables satisfying 
the Markov chain U ^ V X -)■{¥, Zi, Z2) and such that I{U; Y) > I{U] Z2). 

Clearly, the sufficient condition (|T7I) can be obtained from ( |T8i) by choosing V = X and U 
to be a constant. Hence, (fTSi) is a more general sufficient condition than (ITTll . The sufficient 
condition (fTTI) can be proved by considering a nested binning scheme that uses the low-security 
message M2 as part of the transmitter randomness to protect the high-security message Mi 
(when the eavesdropper channel realization is strong). The more general sufficient condition 
(fTSl) can be proved by considering a more complex coding scheme that combines rate splitting, 
superposition coding, nested binning and channel prefixing. A detailed proof of the theorem 
is provided in Sec. 13. 3[ 

The following corollary provides sufficient conditions for establishing that a two-level se- 
curity wiretap channel is (perfectly) embeddable. The conditions are given in terms of the 
existence of a joint auxiliary-input random triple and are immediate consequences of Theo- 
rem [H 

Corollary 2. A two-level security wiretap channel p{y, zi, Z2\x) is embeddable if there exists 
a pair of auxiliary random variables U and V satisfying the Markov chain U ^ V ^ X ^ 
(y, Zi, Z2) and such that I{U ; Y) > I{U ; Z2), p{v, x) is an optimal solution to the maximization 
program (fT5l) . and I(y]Y\U) — I{V;Zi\U) > 0, and it is perfectly embeddable if there exists 
a pair of auxiliary random variables U and V satisfying the Markov chain f/ — > — X — 
{Y, Zi, Z2) and such that I{U ; Y) > I[U ; Z2), p{v, x) is an optimal solution to the maximization 
program f|T5|l . and I{V] Y\U) — I{V] Zi\U) is equal to f|T6|) . 

If, in addition to the Markov chain ( |T2l) . we also have the Markov chain 

X ^Y ^ Z2 (19) 

in that order, the sufficient condition ( fT8l) is also necessary, leading to a precise characterization 
of the secrecy capacity region. The results are summarized in the following theorem; a proof 
of the theorem can be found in Appendix |X1 

Theorem 3. Consider a discrete memoryless two-level security wiretap channel with transition 
probability p{y, zi, Z2\x) that satisfies the Markov chains (fT^ and ffTIJl) . The secrecy capacity 
region of the channel is given by the set of all nonnegative pairs that satisfy flTHl) for some joint 
distribution p{u,v,x), where U and V are auxiliary random variables satisfying the Markov 
chain U X ^ {Y,Zi, Z2). 
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Figure 3: Codebook structure for the nested binning scheme. 



If, in addition to the Markov chains f|T2|) and f|T9|) . we also have the Markov chain 



(20) 



in that order, the (weaker) sufficient condition (fT7|) also becomes necessary, leading to a simpler 
characterization of the secrecy capacity region (which does not involve any auxiliary random 
variables). The results are summarized in the following theorem; a proof of the theorem can 
be found in Appendix [Bl 

Theorem 4. Consider a discrete memoryless two-level security wiretap channel with transition 
probability p{y, zi, Z2\x) that satisfies the Markov chains ( TT2l) . ( |T9l) and ( 120|) . The secrecy 
capacity region of the channel is given by the set of all nonnegative pairs that satisfy ( IT7|) for 
some input distribution p{x) . 



3.3 Proof of Theorem [T] 

We first prove the weaker sufficient condition f lT7|) by considering a nested binning scheme that 
uses the low-security message M2 as part of the transmitter randomness to protect the high- 
security message Mi (when the eavesdropper channel realization is strong). We shall consider 
a random-coding argument, which can be described as follows. 
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Fix an input distribution p{x). 

Codebook generation. Randomly and independently generate 2^^^'^^^^'^'^^ codewords of 
length n according to an ra-product of p{x). Randomly partition the codewords into 2"^^ bins 
so each bin contains 2"'^'^^"'"^) codewords. Further partition each bin into 2"-^^ subbins so each 
subbin contains 2"^ codewords. Label the codewords as x^j^ i where j denotes the bin number, 
k denotes the subbin number within each bin, and / denotes the codeword number within each 
subbin. See Fig. [3] for an illustration of the codebook structure. 

Encoding. To send a message pair (mi, 7712), the transmitter randomly (according to a 
uniform distribution) chooses a codeword t from the subbin identified by (mi, 7722) and 

sends it through the channel. 

Decoding at the legitimate receiver. Given the channel outputs y", the legitimate receiver 
looks into the codebook {x^^k,i}j,k,i and searches for a codeword that is jointly typical [9] with 
y". In the case when 

R1 + R2 + T <I{X;Y) (21) 

with high probability the transmitted codeword t is the only one that is jointly typical 

with I/" (and hence can be correctly decoded). 

Security at the eavesdropper. Note that each bin corresponds to a message mi and contains 
2n(R2+T) codewords, each randomly and independently generated according to an n-product of 
p{x). For a given message mi, the transmitted codeword is randomly and uniformly chosen 
from the corresponding bin (where the randomness is from both the low-security message M2 
and the transmitter's choice of t). Following [3], in the case when 

R2 + T> I{X- Zi) (22) 

we have {l/n)I{Mi\ Z'^) tends to zero in the limit as n 0. Furthermore, each subbin 
corresponds to a message pair (mi,m2) and contains 2'^'^ codewords, each randomly and in- 
dependently generated according to an n-product oip{x). For a given message pair (mi,m2), 
the transmitted codeword is randomly and uniformly chosen from the corresponding subbin 
(where the randomness is from the transmitter's choice of t). Again, following [3], in the case 
when 

T > /(X; Z2) (23) 

we have (l/?7,)/(Mi, M2; Z2) tends to zero in the limit as n — )■ 0. 

Eliminating T from f l2T]) - fl23|) using Fourier-Motzkin elimination, we can conclude that any 
rate pair (i?i,i?2) that satisfies f[T7|l is achievable. 

Next we prove the more general sufficient condition f lTS]) by considering a coding scheme 
that combines rate splitting, superposition coding, nested binning and channel prefixing. We 
shall once again resort to a random-coding argument, which can be described as follows. 

Fix a joint auxihary-input distribution p{u)p{v\u)p[x\v) with /([/; Y) > I{U ; Z2) and e > 0. 
Split the low-security message M2 into two independent submessages M'2 and with rates 
R'2 and i?2, respectively. 

Codebook generation. Randomly and independently generate 2"('^2+^('^'^2)+e) codewords of 
length n according to an ra-product oip{u). Randomly partition the codewords into 2"^2 bins 
so each bin contains 2"(^'^^'-^2)+'') codewords. Label the codewords as u"^. where j denotes 
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Figure 4: Codebook structure for a coding scheme that combines rate sphtting, superposition 
coding and nested binning. 



the bin number, and k denotes the codeword number within each bin. We shall refer to the 
codeword collection {u'^k}j,k as the t/-codebook. 

For each codeword m"^ in the t/-codebook, randomly and independently generate 2"'('^i+^2+^) 
codewords of length n according to an n-product of p{v\u). Randomly partition the codewords 
into 2"^^ bins so each bin contains 2"^'^2'+^-' codewords. Further partition each bin into 2^^''^ 
subbins so each subbin contains 2"-^ codewords. Label the codewords as f^fc / p ^ where (j, k) 
indicates the base codeword u^j^ from which v^f^ i ^^^ was generated, / denotes the bin number, 
p denotes the subbin number within each bin, and q denotes the codeword number within 
each subbin. We shall refer to the codeword collection {Vj'^k,i,p,g}i,p,g ^be l^-subcodebook 
corresponding to base codeword m";,. See Fig. IHfor an illustration of the codebook structure. 

Encoding. To send a message triple (mi, 777,2,7712), the transmitter randomly (according a 
uniform distribution) chooses a codeword u^, from the 7T22th bin in the [/-codebook. Once 
a u^, is chosen, the transmitter looks into the l^-subcodebook corresponding to u^, and 
randomly chooses a codeword ^3 mi m^' ti froiii the subbin identified by (7rii,7n2)- Once a 
v^, ^„ is chosen, an input sequence is generated according to an Tz-product of p{x\v) 
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and is then sent through the channel. 

Decoding at the legitimate receiver. Given the channel outputs y", the legitimate re- 
ceiver looks into the t/-codebook and its l^-codebooks and searches for a pair of codewords 
{u^^k^v'^\k,i,p q) ^h^^ jointly typical [9] with y". In the case when 

R'^ + I{U;Z2) + e < I{U;Y) (24) 
and R1 + R2+T < I{V;Y\U) (25) 

with high probability the codeword pair selection (m"^^ ta' ^mJ, *2 mi mij tj) ^^^^ i^ 

jointly typical [9] with y". 

Security at the eavesdropper. To analyze the security of the high-security message Mi and 
the submessage M2 at the eavesdropper, we shall assume (for now) that both the sub message 
m'g and the codeword selection are known at the eavesdropper. Note that such an 

assumption can only strengthen our security analysis. Given the base codeword u^, the 
encoding of mi and m'g using the corresponding l^-subcodebook is identical to the nested 
binning scheme considered previously (with additional channel prefixing). Thus in the case 
when 

R'^ + T > I{V-Zi\U) (26) 
and T > I{V;Z2\U) (27) 

we have 

-I(Mi;Z^\M') = -J(Mi;Z" M') ^ (28) 
n n 

and -I{Mi,M'';Z^\M') = -I(Mi, M"- Zl}, M') (29) 
n n 

in the limit as n — 00. The equalities in ( l28l) and ( l29l) are due to the fact that (Mi, ) and 
M2 are independent. From (128|) we may conclude that (l/n)J(Mi; Z") — )■ in the limit as 
n — 7- 00. 

To analyze the security of the submessage M2, note that each bin in the f/-codebook 
corresponds to a message m'2 and contains 2'^^^'^^^^^''^'^'>^'^^ codewords, each randomly and 
independently generated according to an n-product of p{u). For a given submessage m^, the 
codeword u^, is randomly and uniformly chosen from the corresponding bin (where the 
randomness is from the transmitter's choice of ^2)- Note from (!27|) that the rate of each 
l^-subcodebook is greater than /(V; Z2\U). Following [lOl Lemma 1], we have 

-/(M^; Zl^) -> (30) 
n 

in the limit as n — )■ 00. Putting together (1291) and (l30l) . we have 

-/(Mi,M2;Z2") = -J(Mi,M^,M^';Z2") 
n n 

= -I{M^;Z^) + -IiMi,M^;Z^\M^) 
n n 
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which tends to zero in the hmit as n — oo. 

Finally, note that the overall communicate rate R2 of the low-security message M2 is given 

by 

Eliminating T, R'^ and R'^ from (I24D-([27D, (jSll), and R'2,R2 > using Fourier- Motzkin elim- 
ination, simplifying the results using the facts that 1) I{U]Y) > I{U;Z2), 2) I{V]Z2\U) < 
I{V; Zi\U) which is due to the Markov chain and 3) I{V- Y\U) + I{U- Y) = I{V, U; Y) = 
I{V;Y) and I{V;Z2\U) + /(f/; Z2) = I{V,U;Z2) = I{V;Z2) which are due to the Markov 
chain U V ^ X [Y, Zi, Z2), and letting e — )■ 0, we may conclude that any rate pair 
(Ri,R2) satisfying f|T8|) is achievable. This completes the proof of Theorem [1] 



4 Gaussian Two-Level Security Wiretap Channels 
4.1 Scalar Channel 

Consider a discrete-time two-level security wiretap channel with real input X and outputs Y, 
Zi and Z2 given by 

Y = v^X + iVi 

Zi = Vb'iX + N2 (32) 
Z2 = VhX + N3 

where a, bi and 62 are the corresponding channel gains, and A'^i, and N3 are additive white 
Gaussian noise with zero means and unit variances. Assume that bi > 62 so the receiver that 
receives the channel output Zi represents a stronger realization of the eavesdropper channel 
than the receiver that receives the channel output Z2. The channel input X is subject to the 
average power constraint 

We term the above communication scenario as (scalar) Gaussian two-level security wiretap 
channel. The following theorem provides an explicit characterization of the secrecy capacity 
region. 

Theorem 5. Consider the (scalar) Gaussian two-level security wiretap channel fl32|) . The 
secrecy capacity region of the channel is given by the collection of all nonnegative pairs -R2) 
that satisfy 

Ri < C,{P,a,b,) 
and R1 + R2 < Cs{P,a,b2) ^ ' 

where Cs{P,a,b) is defined as in ([6]). 

Proof: We first prove the converse part of the theorem. Recall from Sec. 13.11 that without 
transmitting the low-security message M2 (which can only increase the achievable rate Ri ) , the 
problem reduces to communicating the high-security message Mi over the traditional wiretap 
channel p{y,zi\x). For the Gaussian two-level security wiretap channel (132|) . the problem 
reduces to communicating the high-security message Mi over the Gaussian wiretap channel 
with channel outputs Y and Zi given by 

Y = y/EX + Ni 
Zi = Vb~iX + N2. 
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We thus conclude that Ri < Cs{P., a, bi) for any achievable rate Ri. 

Similarly, ignoring the additional secrecy constraint ( !T^ for the high-security message 
Ml (which can only enlarge the achievable rate region {{Ri, R2)}), we can simply view the 
messages Mi and M2 as a single message M with rate Ri + R2- In this case, the problem 
reduces to communicating the message M over the traditional wiretap channel p{y, Z2\x). For 
the Gaussian two-level security wiretap channel fl32l) . the problem reduces to communicating 
the message M over the Gaussian wiretap channel with channel outputs Y and Z2 given by 



We thus conclude that Ri + R2 < Cs{P, a, 62) for any achievable rate pair (i?i, R2). 

To show that any nonnegative pair (i?i,i?2) that satisfies fl33|l is achievable, let us first 
consider two simple cases. First, when 61 > 62 > a, both Cs{P, a, bi) and Cs{P, a, 62) are equal 
to zero (c.f. definition ([6])). So ( l33l) does not include any positive rate pairs and hence there is 
nothing to prove. Next, when bi > a > 62, Cs{P, a, bi) = and ( l33l) reduces to 



Since the high-security message Mi does not need to be transmitted, any rate pair in this 
region can be achieved by using a scalar Gaussian wiretap code to encode the low-security 
message M2. This has left us with the only case with a > 61 > 62- 

For the case where a > 61 > 62, the achievability of any rate pair in ( 133|) follows from that 
of (fT7|) by choosing X to be Gaussian with zero mean and variance P. This completes the 
proof of the theorem. □ 

The following corollary follows directly from the achievability of the corner point 



Corollary 6. Scalar Gaussian two-level security wiretap channels under an average power 
constraint are perfectly embeddable. 

Fig. [5] illustrates the secrecy capacity region (!33|) for the case where a > 61 > 62- Also 
plotted in the figure is the rate region that can be achieved by the naive scheme that uses two 
Gaussian wiretap codes to encode the messages Mi and M2 separately. Note that the corner 
point ( IMl) is strictly outside the "naive" rate region, which illustrates the superiority of nested 
binning over the separate coding scheme. 

4.2 Independent Parallel Channel 

Consider a discrete-time two-level security wiretap channel which consists of a bank of L 
independent parallel scalar Gaussian two-level security wiretap channels. In this model, the 



Y 
Z2 




Ri = 
and R2 < Cs{P,a,b2). 



{R^, R2) = {C,{P, a, 61), C,(P, a, 62) - C,(P, a, b^)) 



(34) 



of ([33]). 
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Figure 5: Secrecy capacity region of the scalar Gaussian two- level security wiretap channel 
(a > 61 > 62)- The rate region under the dashed line can be achieved by separate encoding of 
Ml and M2. 



channel outputs are given hj Y = (Yi, . . . , Yl), Z\ = {Zi^i, . . . , and Z2 = (-Z'2,1, . . . , -Z'2,l) 
where 

Yl = ^Xi + Ni,i 
Zi^i = ^iXi + N2,i / = 1,...,L. (35) 
Z2,i = ^/b^^Xi + N^^i 

Here, Xi is the channel input for the /th subchannel, a;, hi^i and 62,; are the corresponding 
channel gains in the Zth subchannel, and Ni^i, N2^i and N^^i are additive white Gaussian noise 
with zero means and unit variances. We assume that hi^i > 62,; for all ^ = - ■ ■ ,L, so the 
receiver that receives the channel output Zi represents a stronger realization of the eavesdrop- 
per channel in each of the L subchannels than the receiver that receives the channel output 
Z2. Furthermore, {Ni^i, N2^i, N2,^i), I = 1,...,L, are independent so all L subchannels are 
independent of each other. 

We term the above communication scenario as independent parallel Gaussian two-level 
security wiretap channel. The following theorem provides an explicit characterization of the 
secrecy capacity region under an average individual per-subchannel power constraint. 

Theorem 7. Consider the independent parallel Gaussian two-level security wiretap channel 
(!35|) where the channel input X is subject to the average individual per-subchannel power con- 
straint ([H]). The secrecy capacity region of the channel is given by the collection of all nonneg- 
ative pairs (i?i,i?2) that satisfy 

and R1 + R2 < Y.Li^s{Pi,ai,b2,i) 
where Cs{P,a,b) is defined as in ([6]). 
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Proof: We first prove the converse part of tlie tlieorem. Following the same argument as 
that for Theorem [5l we can show that 

Ri < Cs{{Pi,ai,h,i}ti) .^7^ 
and R, + R2 < {{Pi, ai,b2,i} ti) ^ ' 

for any achievable secrecy rate pair {Ri,R2). By the secrecy capacity expression ( fTOl) for the 
independent parallel Gaussian wiretap channel under an average individual per-sub channel 
power constraint, we have 

C,i{Pi,ai,h,i}t,) = Y.liCs{Puauh,{) 
and Cs{{Puauh2AU) = Y:liCs{Pi,ai,b2,i). ^ ' 

Substituting f p8|) into f l37|) proves the converse part of the theorem. 

To show that any nonnegative pair R2) that satisfies fl36|) is achievable, let us consider 
independent coding over each of the L subchannels. Note that each subchannel is a scalar 
Gaussian two-level security wiretap channel with average power constraint Pi and channel 
gains {ai,bi^i,b2,i)- Thus, by Theorem any nonnegative pair (-Ri,;, -R2,/) that satisfies 



Ri,i < Cs{PuaiM,i) 
and Ri^i + R2,i < Cs{Pi,ai,b2,i) 

is achievable for the /th subchannel. The overall communication rates are given by 



(39) 



-Ri - Y.f=iRi 



and R2 = J2f=i P 



(40) 



2A- 



Substituting f l5I?]) into fHU]) proves that any nonnegative pair {Ri,R2) that satisfies f l5B]) is 
achievable. This completes the proof of the theorem. □ 
Similar to the scalar case, the following corollary is an immediate consequence of Theorem[71 

Corollary 8. Independent parallel Gaussian two-level security wiretap channels under an av- 
erage individual per-subchannel power constraint are perfectly embeddable. 

The secrecy capacity region of the channel under an average total power constraint is 
summarized in the following corollary. The results follow from the well-known fact that an 
average total power constraint can be written as the union of average individual per-subchannel 
power constraints, where the union is over all possible power allocations among the subchannels. 

Corollary 9. Consider the independent parallel Gaussian two-level security wiretap channel 
f p5|) where the channel input X is subject to the average total power constraint (|9]). The secrecy 
capacity region of the channel is given by the collection of all nonnegative pair (i?i,i?2) that 
satisfies 

Ri < Y.i=iCs{Phai,bi^i) ^^^^ 
and R1 + R2 < J2i=iCs{Pi,ai,b2,i) 

for some power allocation (Pi, . . . , Pl) such that Xl^i Pi — P ■ 
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Figure 6: Secrecy capacity region of the independent parallel Gaussian two-level security wire- 
tap channel under an average total power constraint. The intersection of the dashed lines are 
outside the secrecy capacity region, indicating that the channel is not perfectly embeddable. 

Fig. [6] illustrates the secrecy capacity with L = 2 subchannels where 

ai = 1.000, = 0.800, 62,1 = 0.100 
a2 = 1.000, 61,2 = 0.250, 62,2 = 0.100 
and P = 1.000. 

As we can see, under the average total power constraint (Q, the independent parallel Gaussian 
two-level security wiretap channel is embeddable but not perfectly embeddable. The reason 
is that the optimal power allocation (Pi, P2) that maximizes Cs{Pi, Oi, 62,1) + Cs{P2, 0,2, ^2,2) is 
suboptimal in maximizing Cs{Pi,ai,bi^i) + Cs(P2, 02, &i,2)- By comparison, under the average 
individual per- sub channel power constraint (jH]), the power allocated to each of the subchannels 
is fixed so the channel is always perfectly embeddable. 

5 Two-Level Security Wiretap Channel II 

In Sec. [2] we briefly summarized the known results on a classical secrecy communication setting 
known as wiretap channel. A closely related classical secrecy communication scenario is wiretap 
channel II, which was first studied by Ozarow and Wyner [5]. In the wiretap channel II setting, 
the transmitter sends a binary sequence X" = of length n noiselessly to an 

legitimate receiver. The signal Z" = [Zi, . . . , Zn) received at the eavesdropper is given by 



where e represents an erasure output, and S" is a subset of {1, . . . ,n} of size na representing 
the locations of the transmitted bits that can be accessed by the eavesdropper. 




Xi, ieS 
e, otherwise 
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If the subset S is known at the transmitter, a message M of n(l — a) bits can be noiselessly 
communicated to the legitimate receiver through Xs^ := {Xi : i G S'^}. Since the eavesdropper 
has no information regarding to Xs^, perfectly secure communication is achieved without any 
coding. It is easy to see that in this scenario, n{l — a) is also the maximum number of bits 
that can be reliably and perfectly securely communicated through n transmitted bits. 

An interesting result of [5] is that for any e > 0, a total of 72(1 — a — e) bits can be reliably and 
asymptotically perfectly securely communicated to the legitimate receiver even when the subset 
S is unknown (but with a fixed size na) a priori at the transmitter. Here, by "asymptotically 
perfectly securely" we mean {l/n)I{M; Z^) — >■ in the limit as n — ?■ oo. Unlike the case where 
the subset 5* is known , coding is necessary when 5" is unknown a priori at the transmitter. In 
particular, [5] considered a random binning scheme that partitions the collection of all length- 
n binary sequences into an appropriately chosen group code and its cosets. For the wiretap 
channel setting, as shown in Sec. |3l a random binning scheme can be easily modified into a 
nested binning scheme to efficiently embed high-security bits into low-security ones. The main 
goal of this section is to extend this result from the classical setting of wiretap channel to 
wiretap channel II. 

More specifically, assume that a realization of the subset S has two possible sizes, nai 
and na2, where 1 > ai > a2 > 0. The transmitter has two independent messages, the high- 
security message Mi and the low-security message M2, uniformly drawn from {1,...,2"^^} 
and {1, . . . , 2"'^2} respectively. When the size of the realization S is na2, both messages Mi 
and M2 need to be secure, i.e., (l/n)/(Mi, M2; Z^) — )■ in the limit as n — > 00. In addition, 
when the size of the realization of S is nai, the high-security message Mi needs to remain 
secure, i.e., (l/n)J(Mi; Z") — )■ in the limit as n — j- 00. We term this communication scenario 
as two-level security wiretap channel II, in line with our previous terminology in Sec. |3l 

By the results of [3], without needing to communicate the low-security message M2, the 
maximum achievable Ri is 1 — ai. Without the additional secrecy constraint (l/n)J(Mi; Z") ^■ 
on the high-security message Mi, the messages (Mi, M2) can be viewed as a single message 
M with rate Ri + R2, and the maximum achievable -Ri -|- -R2 is 1 — 02- The main result of 
this section is to show that the rate pair (1 — ai, ai — ^2) is indeed achievable, from which we 
may conclude that two-level security wiretap channels II are perfectly embeddable. Moreover, 
perfect embedding can be achieved by a nested binning scheme that uses a two-level coset 
code. The results are summarized in the following theorem. 

Theorem 10. Two-level security wiretap channels II are perfectly embeddable. Moreover, 
perfect embedding can be achieved by a nested binning scheme that uses a two-level coset code. 

Proof: Fix e > 0. Consider a binary parity-check matrix 



where the size of Hi is n{l — ai — e) x n and the size of H2 is n{ai — 02) x n. Let si(-) be a 
one-on-one mapping between {1, . . . , 2"(^~"i~'^)} and the binary vectors of length n{l — ai — e), 
and let S2(-) be a one-on-one mapping between M2 G {1, . . . , 2'^("i~"2) j g^j^^j ^j-^g binary vectors 
of length n{ai — 02). 
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For a given message pair (mi, 7712), the transmitter randomly (according to a uniform 
distribution) chooses a solution to the linear equations 



' Hi ' 




si(mi) 


. ^2 _ 




_ S2(m2) _ 



(42) 



and sends it to the legitimate receiver. 

When the parity-check matrix H has full (row) rank, the above encoding procedure is equiv- 
alent of a nested binning scheme that partitions the collection of all length-ra binary sequences 
into bins and subbins using a two-level coset code with parity-check matrices {Hi, H2). More- 
over, let bi, ... ,bn be the columns of H and let F C {1, . . . , n}. Define D2{T) as the dimension 
of the subspace spanned hj {hi : 2 G F} and 

D*^ := min D2(T). 

|r|=n(l-a2) 

When the size of the realization of S is na2, by O Lemma 4] we have 

H{Mi,M2\Z'') = D*2. (43) 

Note that the low-security message M2 is uniformly drawn from {1, . . . ,2^^°'^~°'^^}. So by 
(H2|) . for a given high-security message mi, the transmitted sequence x"' is randomly chosen 
(according to a uniform distribution) as a solution to the linear equations (a;")*iJi = si(mi). 
If we let be the columns of Hi and define 

D: := min Di(T) 

|r|=n(l-ai) 

where -Di(F) is the dimension of the subspace spanned by {a, : « G F}, we have again from [5l 
Lemma 4] 

H{Mi\Z'') = Dl (44) 

when the size of the realization of S is nai. 

Let "^{H) = 1 when we have either H does not have full rank, or D2 < n{l — 02 — e) — 3/e, 
or Dl < 77,(1 — ai — e) — 3/e, and let "^{H) = otherwise. By using a randomized argument 
that generates the entries of H independently according to a uniform distribution in {0,1}, 
we can show that there exists an H with "^{H) = for sufficiently large n (see Appendix ICl 
for details). For such an H, we have from 1^ and (gl]) that (l/n)/(Mi, M2; Z") < 3/(ne) 
when the size of the realization of S is na2, and {l/n)I{Mi; Z") < 3/{ne) when the size of the 
realization of 5" is nai. 

Letting — ?■ 00 and e — (in that order) proves the achievability of the rate pair (1 — 
tti, Q!i — ^2) and hence completes the proof of the theorem. □ 



6 Concluding Remarks 

In this paper we considered the problem of simultaneously communicating two messages, a 
high-security message and a low-security message, to a legitimate receiver, referred to as the 
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security embedding problem. An information-theoretic formulation of the problem was pre- 
sented. With appropriate coding architectures, it was shown that a significant portion of the 
information bits can receive additional security protections without sacrificing the overall rate 
of communication. Key to achieve efficient embedding was to use the low-security message as 
part of the transmitter randomness to protect the high-security message when the eavesdropper 
channel realization is strong. 

For the engineering communication scenarios with real channel input and additive white 
Gaussian noise, it was shown that the high-security message can be embedded into the low- 
security message at full rate without incurring any loss on the overall rate of communication 
for both scalar and independent parallel Gaussian channels (under an average individual per- 
subchannel power constraint). The scenarios with multiple transmit and receive antennas are 
considerably more complex and hence require further investigations. 

Finally, note that even though in this paper we have only considered providing two levels 
of security protections to the information bits, most of the results extend to multiple-level 
security in the most straightforward fashion. In the limit when the security levels change 
continuously, the number of secure bits delivered to the legitimate receiver would depend on 
the realization of the eavesdropper channel even though such realizations are unknown a priori 
at the transmitter. 



A Proof of Theorem 3 



First note that when X Y Z2 forms a Markov chain in that order, we have I{U; Y) > 
I{U; Z2) for any jointly distributed {U,V,X) that satisfies the Markov chain U —> V ^ X ^ 

To show that the sufficient condition ( ITSl) is also necessary, let {Ri,R2) be an achievable 
rate pair. Following Fano's inequality [9] and the asymptotic perfect secrecy constraints (fT3l) 
and (fT4l) . there exists a sequence of codes (indexed by the block length n) of rate pair (i?i, R2) 
such that 

i/(Mi,M2|F") < nej2 (45) 
/(Mi;Zr) < nej2 (46) 
and I{Mi,M2;Z^) < ne„/2 (47) 



where e„ — > in the limit as — )■ 00. 
Following fllS]) and (jlSD, we have 



n{Ri-en) = H{Mi)-nen 

< H{Mi) - [/(Mi; Zr) + H{Mi, Msl^")] 
= i/(Mi|Zf) - H{Mi, Msiy") 

< i7(Mi,M2|Zi") -i/(Mi,M2|y") 
= /(Mi,M2;F")-J(Mi,M2;Z2"). 
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Let M ■= {M,,M2), := (F[l], . . . , F [2 - 1]), Z^^^, = {Z,[t + 1], Zi[n]) and U[i] := 

(Y'-\ We further have 

n(i?i-e„) < I{M;Y")-I{M;Z'^) 

n 

= Yl [/(M;F[z]|r-^) -/(M;Zi[z]|Z,V)] 

i=l 

n 

i=l 
n 

= Y.[i{M-Y[^m]) - i{M;z^mm 

i=l 

= n[I{M;Y[Q]\U[Q],Q) - I{M;Z,[Q]\U[Q],Q)] 

= n [I{M, U[Q], Q- Y[Q]\U[Qi Q) - /(M, U[QlQ- Z,[Q]\U\QIQ)] 

= n[I{V[Q]-Y[Q]\U[QlQ) - I{V[Q]-Z,[Q]\U[QIQ)] 

where (a) is due to the Csiszar-Korner sum equahty [U Lemma 7] , Q is a standard time-sharing 
variable [9], and V[Q] := {M,U[QIQ). 
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Following (H5|) and (H7|) . we have 

n(i?i + i?2 - en) = H{M)-nen 

< H{M) - [i7(M|F") + J(M; Z^)] 
= I{M; F") - /(M; Z^') 



(6) 



[j(M;r[z]|r-i) -/(M;Z2[^]|Z2V)] 

1=1 

n 

i=l 
n 

[HM, Y^-\ zi,^,, zi,^,- Y[^) - i{M, r-\ z,m 

n 

i=l 
n 

-Y[Kzi.^,-,y[m.y'~\zi^^,) - i{zi^,-zM\M,^^^^ 

i=l 



i=l 



< Y [^(M, Y^-\ Zl,^,, Y[^) - J(M, r-\ ZiVi, ^^[z])] 



n 



Y [HM, Y^-\ Zl,^,- - /(Af, ZiVi; Z^)] 



i=l 



5^[J(M,t/[z];r[2])-J(M,f/[^];Z2[2])] 



= n [liM, U[Q];Y[Q]\Q) - /(M, U[Q]; Z,[Q]\Q)] 

= n [I{M, U[Q],Q- Y[Q]) - I{M, U[Q],Q; Z<,\Q\) - {1{Y\Q\- Q) - /(ZaiQ]; Q))] 
= n [I{y[Q]- Y[Q]) - I{V[Q]- Z,[Q]) - {I{Y[Q]- Q) - I{Z,[Q]-Q))] 

< n[I{y[Q]-Y[Q])-I(y[Q]-Z2[Q])] 

where (b) follows from the Csiszar-Korner sum equality [U Lemma 7], (c) is due to the Markov 
chain (fT9l) . (d) is due to the Markov chain (fT2l) . and (e) follows again from the Markov chain 
(fT9|) and the fact that the channel is memoryless. 

Finally, we complete the proof of the theorem by letting U := {U[Q],Q), V := V[Q], 
X := X[Q], Y := Y[Q], Z^ := Zi[Q], Z2 := Z2[Q] and n ^ 00. 



B Proof of Theorem 4 



As shown in Theorem [3l when we have the Markov chains (fT2l) and (JTOll . there exists a random 
triple [U, V,X) satisfying the Markov chain U V ^ X ^ [Y, Zi, Z2) and such that Ri < 
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I{V; Y\U) - I{V; Zi\U) and i?i + i?2 < HV; Y) - I{V; Z2). In fact, the sum rate Ri + R2 can 
be further bounded from above as 



R1 + R2 < I{V,X-Y) - I{V,X;Z2) ~[I{X-Y\V) - I{X-Z2\V)] 
I{X- Y) - I{X- Z,) - [I{X- Y\V) - I{X- Z,\V)] 



where (a) follows from the Markov chain V ^ X ^ {Y, Z2), and (b) follows from the Markov 
chain so I{X; Y\V) > J(X; Z2\V). 



When we further have the Markov chain (120|) . Ri can be further bounded from above as 



Ri < I{U,V;Y) - I{U,V;Zi\U) -[I{U;Y) - I{U;Zi)] 
I{V- Y) - I{V- Z,) - [I{U- Y) - I{U- Z,)\ 

= I{V, X; Y) - I{V, X- Zi) - [/(X; Y\V) - /(X; Z^\V)] 

< I{V,X-Y)-I{V,X-Z^) 
I{X;Y)-I{X;Zi) 



where (c) follows from the Markov chain U ^ V ^ {Y,Zi), (d) and (e) follow from the 
Markov chain (ED]) so /(f/; F) > I{U]Zi) and I{X]Y\V) > I{X]Zi\V), and (f) follows from 
the Markov chain V ^ X ^ {Y, Zi). This completes the proof of the theorem. 



C Existence of an H with ^{H) = 



To show that there exists a parity-check matrix H such that "^{H) = 0, it is sufficient to show 
that E\E'(iJ) < 1 where EX denotes the expectation of a random variable X. 



< I{X;Y)-I{X;Z2) 



Let 



and 




1, A(r) < n(l - - e) - 3/e 
0, otherwise 



1, rank(if) < n(l — 0:2 — e) 
0, otherwise 



for i 



1,2. By the union bound, we have 



E^(if) < E'^oiH) + E^i(if, r). 



(48) 



i=l rc{l,...,n} 

\r\=nai 



By [5l Lemma 6], 



(49) 
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for sufficiently large n. By [5l Lemma 5], for any F C {1, . . . , n} such that |r| = nai 

E^i(if,r) < 2-3"+'^(i-"'-^) < 2~2". 
Since the total number of different subsets of {1, ... , n} is 2", we have 

2 1 

mi{H, r) < 2 • 2" • 2-2" = 2""+^ < - (50) 

1=1 rc{i,...,n} 

|r|=na, 

for n > 2. Substituting f l49p and (150 p into (HSj) proves that E\E'(^r) < 1 for sufficiently large n 
and hence completes the proof. 
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